Skip to content

Don't install source files at the environment root - #792

Closed
Lotion-3 wants to merge 1 commit into
Point72:mainfrom
Lotion-3:fix/wheel-top-level-files
Closed

Lotion-3 wants to merge 1 commit into
Point72:mainfrom
Lotion-3:fix/wheel-top-level-files

Conversation

@Lotion-3

Copy link
Copy Markdown

Fixes #526.

pip install csp puts CMakeLists.txt, cpp/, setup.py, pyproject.toml, vcpkg.json, README.md, LICENSE and NOTICE at the root of the environment (about 290 files).

Cause. MANIFEST.in includes these files for the sdist. Since pyproject.toml has a [project] table, include_package_data defaults to True, and scikit-build then treats every MANIFEST.in entry outside a Python package as a data file. pip installs data files at the environment root.

Change. In setup.py:

  • include_package_data=False, so MANIFEST.in only affects the sdist;
  • package_data lists the package's only non-Python files, the adapter test data in csp/tests/adapters/;
  • packages=find_namespace_packages(include=["csp", "csp.*"]), because csp/build and csp/tests/build have no __init__.py and would be missed by packages=["csp"] or find_packages().

This is the same idea as #647, which failed CI because it dropped csp.build, csp.tests.build and the test data from the wheel. Here the wheel's csp/ contents are unchanged.

Testing. I built wheels from main and from this branch with the same compiled artifacts and compared them:

  • the 295 files at the environment root are gone, and nothing else changed: the 328 files under csp/ are identical, byte for byte;
  • the same holds in a dev checkout with in-place build outputs (csp/lib, csp/include);
  • the sdist file list is unchanged;
  • after pip install, the environment root contains only the usual venv files;
  • make lint passes, and the Python test suite run against the wheel, as CI does (pip install --target . then pytest csp/tests), gives the same results as the released 0.19.1 wheel.

CI. I ran the full CI matrix (ci-full) on my fork. All 15 wheel builds (Linux, Windows and macOS, cp310–cp314), all 15 test jobs, the 3 test_dependencies jobs and test_sdist passed: https://github.com/Lotion-3/csp/actions/runs/43494223844. To get the run through, that branch also had three CI-only changes that aren't part of this PR:

  • macos-14 runners are no longer supported, so I used macos-15 with brew tap homebrew/core --force before brew extract;
  • git.openldap.org returned HTTP 429 for the lmdb source, so I used an overlay port fetching the same LMDB_0.9.33 source from the GitHub mirror;
  • pandas<3.1, since fixed on main by Fix pandas 3.1 compatibility #782.

This branch is rebased on that fix.

I'm happy to send the macOS change as a separate PR if that's useful.

MANIFEST.in includes cpp/, CMakeLists.txt, setup.py, etc. so that the
sdist can be built. Because pyproject.toml has a [project] table,
include_package_data defaults to True, and scikit-build then installs
every MANIFEST.in entry that is outside a Python package as a data
file, which pip puts at the root of the environment.

Turn include_package_data off and list the package's only non-Python
files (the adapter test data) in package_data. Discover packages with
find_namespace_packages so that csp.build and csp.tests.build, which
have no __init__.py, are still included. The files under csp/ in the
wheel are unchanged; the sdist is unchanged.

Fixes Point72#526

Signed-off-by: Laksh Agarwal <lakshagarwal65@gmail.com>
@timkpaine

Copy link
Copy Markdown
Member

Thanks for the PR! I've trimmed the comments and reformatted in #796

@timkpaine timkpaine closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pip installation of csp pulls in source files at top-level of environment

2 participants